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The findings from this review do not refiect the fuii body of research evidence 
on The New Teacher Project Teaching Fellows program. 


What is this study about? 

The study examined whether students taught by 
teachers in the Teach for America (TFA) and The 
New Teacher Project Teaching Feiiows (Teaching 
Feiiows) programs had greater mathematics 
achievement than students taught by teachers who 
were not in either of these programs. This WWC 
report focuses on the Teaching Feiiows intervention, 
and a separate single study review provides infor- 
mation about the TFA intervention. 

The Teaching Feiiows study included two cohorts 
of students in grades 6-1 2 (one from the 2009-1 0 
school year and one from the 201 0-1 1 school year). 
Students came from eight states, nine school districts. 


Features of The New Teacher Project 
Teaching Fellows (Teaching Fellows) 


Teaching Fellows is a highly selective alternative 
certification program for new teachers. The program 
recruits, selects, trains, places, and provides support 
to teachers in low-income, high-need schools 
across the country. 

Teaching Fellows recruits both new college graduates 
and professionals who would like to switch careers. 
Training is intended to provide over 68 hours of 
instruction and 64 hours of fieldwork during a summer 
institute, and most Teaching Fellows teachers also 
attend state-authorized alternative certification 
programs. Teaching Fellows teachers are asked to 
make an open-ended commitment to teaching. 


WWC Rating 


The research described in this 
report meets WWC group design 
standards without reservations 

The Teaching Fellows study is a well-executed 
randomized controlled trial with low levels of 
sample attrition."* 


and 44 schools. Within each school, students were 
randomly assigned to a mathematics classroom 
taught by either a Teaching Feiiows teacher or a 
comparison teacher who did not enter teaching 
through a highly selective alternative route. In total, 
the Teaching Feiiows study randomly assigned over 
7,000 students to either a Teaching Feiiows teacher 
or a comparison teacher. 

Mathematics achievement was assessed for students 
in grades 6-8 using scores from state-required assess- 
ments. For students in grades 9-12, the authors 
administered end-of-course math assessments devel- 
oped by the Northwest Evaluation Association (NWEA). 

What did the study find? 

The study authors found, and the WWC confirmed, 
that Teaching Feiiows teachers did not differ in 
effectiveness from comparison teachers on improv- 
ing student mathematics achievement scores.® 
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Appendix A: Study details 

Clark, M. A., Chiang, H. S., Silva, T., McConnell, S., Sonnenfeld, K., Erbe, A., & Puma, M. (2013). 

The effectiveness of secondary math teachers from Teach For America and the Teaching 
Feiiows programs (NCEE 2013-4015). Washington, DC: National Center for Education Evaluation 
and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. 


Setting 

The New Teacher Project Teaching Fellows (Teaching Fellows) study was conducted in eight 


states, nine school districts, and 44 schools. 

Study sample 

In each participating school, researchers matched two or more math classrooms so that at 
least one classroom would be taught by a Teaching Fellows teacher, and at least one class- 
room would be taught by a teacher who did not participate in a highly selective alternative 
route to certification (such as Teaching Fellows or TFA). Students in grades 6-12 were ran- 
domly assigned to these classrooms, typically at the beginning of the school year. However, 
students who entered a participating school after the beginning of the school year were 
randomly assigned to classrooms through rolling random assignment. The Teaching Fellows 
study was conducted with two cohorts of students in the 2009-10 and 2010-1 1 school years. 
The Teaching Fellows study sample was comprised primarily of high school classrooms (69%). 

The Teaching Fellows study was conducted in 44 schools located in eight states, with 118 
classroom matches covering 69 intervention and 84 comparison teachers. For this study, 

3,659 students were randomly assigned to a Teaching Fellows teacher, and 3,629 students 
were randomly assigned to a comparison teacher. The analytic sample included 2,127 students 
in the intervention group and 1 ,989 students in the comparison group. 

Intervention 

group 

The intervention group consisted of classrooms of Teaching Fellows teachers. Teaching Fellows 
recruits, selects, trains, places, and provides support to teachers in low-income, high-need 
schools across the country. The Teaching Fellows program recruits both new college gradu- 
ates and professionals who would like to switch careers. The program is highly selective 
(Teaching Fellows accepts 13% of applicants), and recruits apply to programs located in the 
region where they intend to teach. The application process for the Teaching Fellows program 
involves a written application, an analysis and writing exercise, an in-person interview, a moni- 
tored group discussion, and a 5-minute sample teaching lesson. Once selected. Teaching 
Fellows teachers perform 25 hours of independent study, participate in a summer institute, 
and participate in a local certification program. The summer institute consists of 68 hours of 
instruction and 64 hours of fieldwork. Teaching Fellows teachers are asked to make an open- 
ended commitment to teaching. 

Comparison 

group 

Classrooms were eligible to be in the comparison group if their teachers were certified through 
a route other than TFA, Teaching Fellows, or a similar program. 
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Outcomes and 
measurement 


Support for 
implementation 


Reason for 
review 


For students in grades 6-8, the authors measured mathematics achievement using scores 
obtained on state- required assessments. For students in grades 9-12, the authors admin- 
istered end-of-course mathematics assessments developed by the Northwest Evaluation 
Association (NWEA). All test scores were converted to z-scores to facilitate comparisons 
across classrooms, schools, and districts. For a more detailed description of these outcome 
measures, see Appendix B. 

Teaching Fellows recruits are required to (a) complete 25 hours of independent study and a 
4-hour orientation meeting and (b) attend a 5-7 week summer institute that includes 68 hours 
of instruction and an average of 64 hours of fieldwork via 15 5-hour days in a summer school 
classroom. In addition, most teachers attend state-authorized alternative certification programs. 
During the school year. Teaching Fellows teachers are provided information, feedback, and 
mentoring through the Teaching Fellows program. 

This study was identified for review by the WWC by receiving media attention. 
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Appendix B: Outcome measures for the mathematics achievement domain 


Mathematics achievement 


Mathematics achievement 


Mathematics assessments 

For students in grades 6-8, the authors measured mathematics achievement using scores from state-required 
assessments. These assessments were administered at the end of the schooi year in which the students were 
randomiy assigned. 

For students in grades 9-12, the authors administered end-of-course mathematics assessments deveioped 
by the Northwest Evaiuation Association (NWEA). These computer-adaptive assessments covered generai high 
schooi math, Aigebra i. Geometry, or Aigebra il, depending upon the course content. 

Aii test scores were converted to z-scores. Eor state assessments administered to middie schooi students, 
the reference popuiation was the fuii popuiation of students in the same state, year, and grade who took the 
same assessment. Eor NWEA end-of-course assessments administered to high schooi students, the reference 
popuiation was the NWEA’s nationwide norming sampie for that assessment. 
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Appendix C: Study findings for the mathematics achievement domain, Teaching Fellows study 


Mean 

(standard deviation) WWC calculations 


Domain and 
outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Mathematics achievement 

Mathematics assessments 

Cohorts 
1 and 2 

44 schools/ 
153 teachers/ 
4,116 students 

nr 

nr 

nr 

0.00 

0 

0.96 

Domain average for mathematics achievement 




0.00 

0 

Not 

statistically 

significant 


Table Notes: For mean difference, effecf size, and improvement index vaiues reported in the tabie, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on student outcomes, representing the average change expected for aii students 
who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an aiternate presentation of the effect size, refiecting the 
change in an average student's percentiie rank that can be expected if the student is given the intervention. The statisticai significance of the study's domain average was deter- 
mined by the WWC. nr = not reported. 

Study Notes: No corrections for ciustering or muitipie comparisons and no difference-in-differences adjustment were needed. The p-vaiue presented here was reported in the 
originai study. The effect size is the impact estimate from the study because the outcome was scaied to be in standard deviation units. This study is characterized as having an 
indeterminate effect because the effect for this intervention was neither statisticaiiy significant or substantiveiy important. For more information, piease refer to the WWC Stan- 
dards and Procedures Flandbook, version 3.0, pages 26-27. 
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Endnotes 

■' Single study reviews examine evidence published in a study (supplemented, if necessary, by information obtained directly from the 
author[s]) to assess whether the study design meets WWC evidence standards. The review reports the WWC’s assessment of whether 
the study meets WWC evidence standards and summarizes the study findings following WWC conventions for reporting evidence on 
effectiveness. This study was reviewed using the single study review protocol, version 2.0. A quick review of this study was released 
on November 7, 2013, and this report is the follow-up review that replaces that initial assessment. 

^ Absence of conflict of interest: This study was conducted by staff from Mathematica Policy Research. Therefore, Mathematica 
reviewers were not involved in the WWC review of this study. 

® The study also examined the impact of Teach for America relative to a separate comparison group that was formed by random 
assignment. The findings from that analysis are reported in a separate single study review by the WWC because the impacts of the 
two programs were measured separately. The authors also noted that the study was not designed to compare the effectiveness of 
Teach For America relative to Teaching Feliows. 

The WWC verified low study attrition through an author query. 

® The study also examined impacts by subgroups including school level (e.g., middle and high) and by type of comparison teacher 
(e.g., traditionally certified and alternatively certified). However, the study did not provide enough information to determine a rating for 
these analyses, so they are not presented in this single study review. 

Recommended Citation 

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2014, May). 

WWC review of the report: The effectiveness of secondary math teachers from Teach for America 
and the Teaching Fellows programs. Retrieved from http://whatworks.ed.gov 
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Glossary of Terms 

Attrition 


Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Improvement index 


Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Single-case design 
(SCD) 

Standard deviation 


Statistical significance 
Substantively important 


Attrition occurs when an outcome variable is not avaiiabie for aii participants initiaiiy assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review if it falls within the scope of the review protocol and uses either 
an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

Along a percentile distribution of students, the improvement index represents the gain 
or loss of the average student due to the intervention. As the average student starts at 
the 50th percentile, the measure ranges from -50 to +50. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which subjects are assigned 
to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which investigators randomly assign 
eligible participants into intervention and comparison groups. 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 

The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample are spread out over a large range of values. 

Statistical significance is the probability that the difference between groups is a result of 
chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < 0.05). 

A substantively important finding is one that has an effect size of 0.25 or greater, regardless 
of statistical significance. 


Please see the WWC Procedures and Standards Handbook (version 3.0) for additional details. 
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